DeepLearningKit - an GPU Optimized Deep Learning Framework for Apple's iOS, OS X and tvOS developed in Metal and Swift

In this paper we present DeepLearningKit an open source framework that supports using pretrained deep learning models (convolutional neural networks) for iOS, OS X and tvOS. DeepLearningKit is developed in Metal in order to utilize the GPU efficiently and Swift for integration with applications, e.g. iOS-based mobile apps on iPhone/iPad, tvOS-based apps for the big screen, or OS X desktop applications. The goal is to support using deep learning models trained with popular frameworks such as Caffe, Torch, TensorFlow, Theano, Pylearn, Deeplearning4J and Mocha. Given the massive GPU resources and time required to train Deep Learning models we suggest an App Store like model to distribute and download pretrained and reusable Deep Learning models. 1 GPU Accelerated Deep Learning Library The Metal programming language is most the efficient way of utilizing the GPU on Apple’s iOS since 2014 [1, 2, 3, 4] and OSX since 2015 [5, 6, 7]. This paper gives a brief overview of a Metal and Swift based Deep Learning library named DeepLearningKit, in particular parts of Metal convolutional neural network operators for the GPU. DeepLearningKit supports on-device Deep Learning on Apple’s iOS, OS X and tvOS. DeepLearningKit currently has shader functions for convolutional neural networks implemented in Metal and parallelized for the GPU, operators include: convolution, pooling, rectifier layer and softmax. In terms of deep learning model supported it has support for Min Lin’s Caffe-trained Network In Network[8] (NIN trained on CIFAR-10, CIFAR-100 and ImageNet data sets). We also have preliminary support running Theano[9] trained LeNet (trained on MNIST digit classification dataset). The reason we have chosen NIN is that the network is small compared to other deep convolutional neural networks, but at the same time provide very high classification accuracy on images, e.g. better than AlexNet. GoogleLeNet (winner of Imagenet 2014) uses a similar approach as NIN[?]. NIN can perhaps also be used in non-image domains, e.g speech recognition[10] or natural language processing[11]. In particular one could attempt to adapt Zhang and Lecun’s encoding and 1D convolutional operators in “Text Understanding from Scratch”[12] and use it with NIN. 1.1 Experiences with PowerVR G6430/GT7600 on iPhone 5S/6S The performance of DeepLearningKit Deep Learning going from iPhone 5S (with PowerVR G6430 according to The iPhone 5S Review (AnandTech)) to iPhone 6S (with PowerVR GT7600 according to Apple iPhone 6S Plus vs. Samsung Galaxy S6 Edge+) we got 1 order of magnitude in improved performance. Calculation time to run through a 20 layer deep convolutional neural network model ∗http://DeepLearningKit.org 1 ar X iv :1 60 5. 04 61 4v 1 [ cs .L G ] 1 5 M ay 2 01 6